Methods and systems for 3D object detection using learning

ABSTRACT

In a method of 3D object detection, a learning procedure is used for feature selection from a feature set based on an annotated image-volume database, generating a set of selected features. A classifier is built using a classification scheme to distinguish between an object location and a non-object location and using the set of selected features. The classifier is applied at a candidate volume to determine whether the candidate volume contains an object of interest.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims the benefit of U.S. Provisional Application Ser.No. 60/607,601 , filed Sep. 7, 2004 and entitled “Boosting-Based 3DObject Detection with Application to Tumor Detection in 3D Lung Images,”the content of which is hereby incorporated by reference in itsentirety. This application further claims the benefit of U.S.Provisional Application Ser. No. 60/616,357 , filed Oct. 6, 2004 andentitled “Boosting Incorporating Feature Cost,” the content of which ishereby incorporated by reference in its entirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to methods and systems for 3D objectdetection. More particularly, the present invention relates to methodsand systems for 3D object detection using learning.

2. Description of the Related Art

Detection of objects is important in medical and non-medicalapplications. For example, medical image analysis and diagnosis dependson the ability to detect anatomical structures. FIG. 1 shows examples oflung tumors in a 3D CT lung image.

Object detection in 3D and 4D (4D=3D+Time) data is difficult given thelarge amount of data associated with each image, increased noise leveland computational complexity. In object detection, the classes to bediscriminated are not defined by the variations of the different objectsthemselves, but rather by distinguishing between “images containing theobject” and “images not containing the object.” Without restricting thedomain of images for which the system must discriminate, the task oftraining an object detection system is time-consuming and difficult.

Machine learning, an area of artificial intelligence concerned with thedevelopment of techniques that allow computers to “learn” through theanalysis of data sets, investigates the mechanisms by which knowledge isacquired through experience. The field of machine learning is concernedwith both the analysis of data and the algorithmic complexity ofcomputational implementations. Machine learning has a wide spectrum ofapplications including: stock market analysis, classifying DNAsequences, search engines, speech and handwriting recognition, medicaldiagnosis, and game playing. Common learning algorithm types includesupervised learning or classification, unsupervised learning andreinforcement learning.

In machine learning theory, combining multiple classifiers is aneffective technique for improving prediction accuracy. There arenumerous general combining algorithms such as bagging, boosting, anderror-correcting output codes. Boosting, which has its roots in PAC(probably approximately correct) learning, is a machine learningalgorithm for performing supervised learning. Freund, Y. and Schapire,R. E. (1996), “Experiments with a new boosting algorithm,” In MachineLearning: Proceedings of the Thirteenth International Conference, Bari,Italy, pp. 148-156.

The idea of boosting is to design a series of training sets and use acombination of classifiers trained on these sets. The training sets arechosen sequentially, with the weights for each training example beingmodified based on the success of the classifier trained on the previousset. That is, greater weight is assigned to those training examples thatwere difficult to classify, i.e., those for which the misclassificationrate was high, and lower weights to those that were easy to classify.Note that boosting can also be applied to learning methods that do notexplicitly support weights. In that case, random sub-sampling can beapplied to the learning data in the successive steps of the iterativeboosting procedure. Freund and Schapire's AdaBoost algorithm isgenerally considered as a first step towards more practical boostingalgorithms. Freund, Y. and Schapire, R. E. (1997), “A decision-theoreticgeneralization of online learning and an application to boosting,” InJournal of Computer and System Sciences, 55(1): 119-139.

As discussed in U.S. Pat. No. 6,546,379, boosting refers to a family ofgeneral methods that seek to improve the performance obtained from anygiven underlying method of building predictive models by applying theunderlying methods more than once and then combining the resulting“weak” models into a single overall model that, although more complexthan any of the “weak” models obtained from the underlying method, maymake more accurate predictions. The term “weak”, as used in connectionwith boosting, is a technical term used in the art; a “weak” model hasimperfect performance that one hopes to improve by somehow combining theweak model with other weak models built by the same underlying method,but from different training examples of the available training sets.U.S. Pat. No. 6,546,379, entitled “Cascade boosting of predictivemodels,” issued on Apr. 8, 2003 to Hong et al.

The task of classification is a key component in the fields of computervision and machine learning. When an object is presented to a system forclassification, the system selects specific features from the object andthese features are passed to the classifier. The size of the searchspace grows exponentially with respect to the number of features; it isimpossible to search exhaustively in the hypothesis space to find theoptimal classifier.

Feature selection is an important part of many machine learningproblems. Feature selection is used to improve the efficiency oflearning algorithms by finding an optimal subset of features. FIG. 2illustrates sample features for object detection with differentcomputational costs. There has been a great deal of work on developingfeature selection methods. Liu and Motoda provide an overview of themethods developed since the 1970s. Liu, H., and Motoda, H. (1998),Feature Selection for Knowledge Discovery and Data Mining, ISBN0-7923-8198-X, Kluwer Academic Publishers.

The principal aim of designing a classifier is to accurately classifyinput. Classification accuracy depends on diverse factors, includingsample size and the quality of training data. Duda, R., Hart, P. andStork, D. (2001), Pattern Classification, 2^(nd) ed., ISBN:0471-05669-3, John Wiley & Sons. Research has shown two approaches,namely, the support vector machine approach and boosting, are effectivefor classification. Meir and Ratsch provide an overview of applicationsof boosting algorithms; the main ideas are illustrated on the problem ofbinary classification. Meir, R. and Ratsch, G (2003), “An introductionto boosting and leveraging,” In Advanced Lectures on Machine Learning,pp. 119-184, Springer.

Classification accuracy is a critical design consideration, as discussedabove. Many real-time computer vision tasks also require a fastprocessing response, and this requires fast classification processes.For example, methods to detect faces in videos require scanning througha large number of possible candidate regions. Another more challengingexample is the task of tumor detection in 3D CT lung images, whichrequires large 3D sub-volume datasets for scanning. Classificationaccuracy and fast classification processes are critical in developing 3Dobject detection methods and systems for medical and non-medicalapplications.

SUMMARY OF THE INVENTION

According to an exemplary embodiment of the present invention, a methodis provided for 3D object detection. The method includes using alearning procedure for feature selection from a feature set based on anannotated image-volume database, generating a set of selected features;building a classifier using a classification scheme to distinguishbetween an object location and a non-object location and using the setof selected features; and applying the classifier at a candidate volumeto determine whether the candidate volume contains an object ofinterest.

According to an exemplary embodiment of the present invention, aboosting method incorporating feature cost to simultaneously optimizedetection performance and cost is provided. The method includes thesteps of: (a) inputting a set of training samples having a set offeatures and feature computational costs for the set of features; (b)generating a set of weak learners based on the set of features; and (c)building a boosting classifier based on a subset of the weak learners,to minimize a cost function of the detection performance and the featurecomputational costs.

According to an exemplary embodiment of the present invention, aboosting method incorporating feature cost to simultaneously optimizedetection performance and cost is provided. The method includes thesteps of: (a) inputting a set of training samples having a set offeatures, feature computational costs for the set of features, and apre-determined number L that denotes the maximal number of strongclassifiers; (b) generating a set of weak learners based on the set offeatures; (c) initializing a set of final strong classifiers (FSCs) anda set of candidate strong classifiers (CSCs), the set of FSCs beinginitialized to null, the set of CSCs being initialized to contain theset of weak learners; (d) applying a selection criterion, selecting thebest CSCs from the set of CSCs, generating a next set of CSCs; (e)updating the set of FSCs; (f) if the magnitude of the set of FSCs isequal to L, performing an output module; and (g) using a childrenfunction, generating at least one new CSC, adding the at least one newCSC to the next set of CSCs.

According to an exemplary embodiment of the present invention, a methodis provided for detection of tumors in 3D CT lung images. The methodincludes using a learning procedure for feature selection from a featureset based on an annotated image-volume database, generating a set ofselected features; building a classifier using a classification schemeto distinguish between a tumor location and a non-tumor location andusing the set of selected features; and applying the classifier at acandidate volume to determine whether the candidate volume contains atumor.

According to an exemplary embodiment of the present invention, acomputer readable medium including computer code for 3D object detectionis provided. The computer readable medium includes: computer code forusing a learning module for feature selection from a feature set basedon an annotated image-volume database, generating a set of selectedfeatures; computer code for building a classifier using a classificationscheme to distinguish between an object location and a non-objectlocation and using the set of selected features; and computer code forapplying the classifier at a candidate volume to determine whether thecandidate volume contains an object of interest.

According to an exemplary embodiment of the present invention, acomputer system for 3D object detection is provided. The computer systemincludes a processor; and computer program code that executes on theprocessor. The computer program code includes: computer code for using alearning module for feature selection from a feature set based on anannotated image-volume database, generating a set of selected features;computer code for building a classifier using a classification scheme todistinguish between an object location and a non-object location andusing the set of selected features; and computer code for applying theclassifier at a candidate volume to determine whether the candidatevolume contains an object of interest.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will become more apparent to those of ordinaryskill in the art when descriptions of exemplary embodiments thereof areread with reference to the accompanying drawings, of which:

FIG. 1 shows examples of lung tumors in a 3D CT lung image.

FIG. 2 illustrates sample features for object detection with differentcomputational costs.

FIG. 3 illustrates ten types of 3D features for tumor detection,according to exemplary embodiments of the present invention.

FIG. 4 illustrates types of 3D features for object detection, accordingto exemplary embodiments of the present invention.

FIG. 5 illustrates three types of 3D features for object detection,according to exemplary embodiments of the present invention.

FIG. 6 shows an example of a pre-scanned 2D plane within a 3D volume,according to exemplary embodiments of the present invention.

FIG. 7A illustrates a 3D integral image, according to exemplaryembodiments of the present invention.

FIG. 7B illustrates an exemplary set of 2D integral images correspondingto the 3D integral image shown in FIG. 7A.

FIG. 8 is a flowchart showing a boosting method incorporating featurecost, in accordance with an exemplary embodiment of the presentinvention.

FIG. 9 is a flowchart showing a boosting method incorporating featurecost, in accordance with an exemplary embodiment of the presentinvention.

FIG. 10 is a flowchart showing a method of 3D object detection, inaccordance with an exemplary embodiment of the present invention.

FIG. 11 is a flowchart showing a method of detection of tumors in 3D CTlung images, in accordance with an exemplary embodiment of the presentinvention.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

Hereinafter, the present invention will be described in detail withreference to the accompanying drawings.

FIG. 3 illustrates ten types of 3D features for tumor detection,according to exemplary embodiments of the present invention. FIG. 4illustrates exemplary types of 3D features for object detection,according to exemplary embodiments of the present invention. It shouldbe understood that other types of 3D features can be used to implementthe invention. For example, additional types of 3D features for objectdetection, according to exemplary embodiments of the present invention,are illustrated in FIG. 5. It will be understood that the adoption ofdifferent types of 3D features depends on the application and/or systemconstraints.

Referring to FIG. 3, the ten types of 3D features for tumor detectionare used, according to exemplary embodiments of the present invention,for tumor detection in 3D CT lung images. The feature value f for thefeature Type 10 shown in FIG. 3 is computed as:

$f = {\frac{S_{{inside}\mspace{14mu}{cube}}}{n_{{inside}\mspace{14mu}{cube}}} - \frac{S_{{shell}\mspace{14mu}{region}}}{n_{{shell}\mspace{14mu}{region}}}}$

Generally, cancer tumors are rare events in the lungs; therefore, alarge amount of negative samples have to be provided to the learningmethod for training the classifier. According to an embodiment of thepresent invention, the source of the negative samples is the set of thesub-images containing no tumor in the 3D CT lung image. Hereinafter, atraining procedure applying a bootstrap strategy, in accordance withexemplary embodiments of the present invention, will be described.

Training procedure applying a bootstrap strategy Input: 3D CT lungimages and the ground-truth tumor location in these images; and themaximal number of negative samples for training n Do 1. P = {the set oftumor images}; randomly select a set of non-tumor images N_(i) 2. i ← 03. Implement leaming procedure using P and N_(i), obtaining strongclassifier SC_(i) 4. i ← i + 1, N_(i) ← null 5. While |N_(i)| < n a)Scan a non-tumor sub-image I_(neg) in the 3D CT lung images, and${use}\mspace{14mu}\overset{i}{\bigcap\limits_{j = 0}}\mspace{11mu}{{SC}_{j}\mspace{14mu}{to}\mspace{14mu}{classify}\mspace{14mu}{them}}$b) If I_(neg) is classified as positive, N_(i) = N_(i) U I_(neg) c) If |N_(i) | = n or no more non-tumor sub-image to scan, go to 6 end while 6.If | N_(i) | = 0, terminate; otherwise, go to 3 Output: a series ofstrong classifiers {SC₀, SC₁, SC₂, . . . , . . . }

In accordance with exemplary embodiments of the present invention,multiple strong classifiers are derived by using the training procedureapplying the bootstrap strategy, as described above.

FIGS. 2A through 2C illustrate sample features for object detection withdifferent computational costs. For the 3D feature illustrated in FIG.2C, the feature extraction requires 12 array reference accesses. In the2D case, as shown in FIG. 2B, the feature extraction requires 6 arrayreference accesses. To reduce computational efforts, according toexemplary embodiments of the present invention, a 3D integral volume isdefined that is computed once for each processed volume. According to atleast one embodiment of the present invention, intermediate 2D integralplanes are also computed inside the 3D volume. It will be understoodthat the 2D planes can be oriented in either the x-y, x-z, or y-z plane.FIG. 6 shows an example of a pre-scanned 2D plane within a 3D volume,according to exemplary embodiments of the present invention.

According to an embodiment of the present invention, a pruning 2Dpre-scan is performed. In the pruning 2D pre-scan, a 2D plane is scannedwithin a 3D volume prior to the classification using its 3D features. Inthe pruning 2D pre-scan, according to at least one embodiment of thepresent invention, fast 2D detectors are applied. Preferably, the fast2D detectors have a low false-negative rate.

FIG. 7A illustrates a 3D integral image, according to exemplaryembodiments of the present invention. Referring to FIG. 7A, the value at(x, y, z) is the summation of all pixels within the small cube. FIG. 7Billustrates an exemplary set of 2D integral images corresponding to the3D integral image shown in FIG. 7A. The process of computing the 2Dintegral image and 3D integral volume, according to exemplaryembodiments of the present invention, is illustrated in FIGS. 7A and 7B.Its computational complexity is O(n₁, n₂, n₃), where n₁, n₂, n₃ are thewidth, height, and depth of the 3D volume, respectively. The 3D integralimage shown in FIG. 7A can be obtained by directly summing up the 2Dintegral images shown in FIG. 7B. To compute the 2D features, theintegral image for each 2D plane is needed. This does not increase thecomputational load because the 2D integral images, according to at leastone embodiment of the present invention, can be obtained during thecomputation of the 3D integral volume.

To accelerate the detection, in accordance with exemplary embodiments ofthe present invention, a cascade of classifiers (or a pruning tree) canbe constructed. The details concerning one example of a cascade ofclassifiers is disclosed in co-owned U.S. Provisional Application Ser.No. 60/551,585 , filed Mar. 9, 2004 and entitled “Boosted Cascade withMemory,” the content of which is hereby incorporated by reference in itsentirety.

To provide context for the discussion herein of classifier designs inaccordance with embodiments of the present invention, it is noted thatthe AdaBoost algorithm (Freund and Schapire) searches the weakhypotheses in a greedy mode, i.e., the sample distribution is adaptiveto the current weak hypothesis. This is actually a depth-first searchstrategy. Accordingly, the (AdaBoost algorithm) search is easilydiverted from the optimal solution in the iterative boosting procedure

Hereinafter, classifier designs, according to exemplary embodiments ofthe present invention, will be described. In a boosting-based classifierdesign, according to an embodiment of the present invention, the weakhypothesis is balanced between breath-first and depth-firstrequirements. To balance the weak hypothesis between breath-first anddepth-first requirements, according to an embodiment of the presentinvention, a tunable tradeoff between explorative and greedy mode isused. At each boosting iteration, a new weak hypothesis with its weightis determined and added into the strong classifier. This process can beconceptualized as building a search route, with a new node(corresponding to the newly-added weak hypothesis) being added to theroute at each iteration.

According to an embodiment of the present invention, multiple searchroutes are built and stored in memory during the boosting process. Dueto limitations of computer memory, it is generally impossible to storeall the search routes during the search. This computer memory constraintmay be denoted by L, the value of which represents the maximal number ofclassifiers stored in memory.

Hereinafter, a boosting method incorporating feature cost will bedescribed. To guarantee that the boosting search is implemented in thewhole weak hypothesis space, and to avoid being stuck into the localminima prematurely, according to exemplary embodiments of the presentinvention, the selected weak hypotheses are each made different from theothers. The difference between two weak hypotheses can be measured bythe overlap between the two sample sets which are correctly classifiedby the two respective weak hypotheses.

A boosting method incorporating feature cost, according to exemplaryembodiments of the present invention, is presented below.

Boosting method incorporating feature cost Input: • The training samplesX = {(x₁,y₁), . . . ,(x₁,y₁)}, which have d features • The computationalcosts for the d features W = {w₁,w₁, . . . ,w_(d)} • The maximal numberof classifiers allowed to be memorized by system L Initialization: F ←null ψ ← all the d weak learners t ← 0 while true do t ← t + 1 Φ ←Select_(L−|F|)(ψ|W_(i)X) for each φ_(i) ε Φ do if Criterion_(terminate)(φ_(i)|X) = true(φ_(i) ε Φ) then F ← F ∪ {φ_(i)} Remove φ_(i) from Φ endif end for If |F| = L, go to Output ψ ← null If |F| = L, go to Output ψ← null for each φ_(i) ε Φ do ψ ← ψ ∪ (φ_(i) + Children (φ_(i)|t,W,X))end for end while Output: F_(output) (x) = arg min_(FεF)Cost(F|W,X)

FIG. 8 is a flowchart showing a boosting method incorporating featurecost, in accordance with an exemplary embodiment of the presentinvention. Referring to FIG. 8, in a step 810, inputs include a set oftraining samples having a set of features, and feature computationalcosts for the set of features. The feature set may include 3D features,2D features, 1D features, complex features, and/or combinations thereof.

In a step 820, a set of weak learners is generated based on the set offeatures. In a step 830, a boosting classifier is built based on asubset of the weak learners to minimize a cost function of the detectionperformance and the feature computational costs. Detection performanceincludes, but is not limited to, classification accuracy.

FIG. 9 is a flowchart showing a boosting method incorporating featurecost, in accordance with an exemplary embodiment of the presentinvention. FIG. 10 is a flowchart showing a method of 3D objectdetection, in accordance with an exemplary embodiment of the presentinvention. Hereinafter, a method of 3D object detection, in accordancewith exemplary embodiments of the present invention, will be describedwith reference to FIGS. 9 and 10.

Referring to FIG. 10, in a step 1010, a learning procedure is used forfeature selection from a feature set based on an annotated image-volumedatabase, generating a set of selected features. The feature set mayinclude 3D features, 2D features, 1D features, complex features, and/orcombinations thereof.

According to at an exemplary embodiment of the present invention, thestep 1010 is performed using a feature selection method. Preferably, thefeature selection method is a boosting method. The boosting method, inaccordance with exemplary embodiments of the present invention,incorporates feature cost to simultaneously optimize detectionperformance and cost. Detection performance includes, but is not limitedto, classification accuracy. Cost includes, but is not limited to,feature acquisition cost or feature computational cost.

Referring to FIG. 9, in a step 905, the inputs to the boosting methodinclude a set of training samples having a set of features, featurecomputational costs for the set of features, and a pre-determined numberL that denotes the maximal number of strong classifiers.

In a step 910, a set of weak learners is generated based on the set offeatures. In a step 920, a set of final strong classifiers (“FSCs”) isinitialized to null, and a set of candidate strong classifiers (“CSCs”)is initialized to contain the set of weak learners.

In a step 930, a selection criterion is used to select the best CSCsfrom the set of CSCs, generating a next set of CSCs. The next set ofCSCs is stored in memory, in a step 940. For each CSC in the next set ofCSCs, if a termination criterion is not satisfied, the CSC is added tothe set of FSCs and removed from the next set of CSCs, in a step 950.Preferably, the termination criterion is based on a validation setand/or a negligible improvement on a class separation margin using atraining set.

In a step 960, it is determined if the magnitude of the set of FSCs isequal to L. In the case when the magnitude of the set of FSCs is equalto L, an output module is performed, in a step 965. In the outputmodule, according to exemplary embodiments of the present invention, thebest strong classifier is selected from the set of FSCs based on abalancing method. Preferably, the balancing method comprises balancingclassification accuracy and feature cost. Feature cost refers to featureacquisition cost and/or feature computational cost.

In the case when the magnitude of the set of FSCs is not equal to L, thenext set of CSCs is re-initialized to null, in a step 970. In a step980, for each the set of CSCs a children function is used to generateone or more new CSC that are added to the next set of CSCs. In anembodiment of the present invention, the children function is used toselect one or more weak learners from the next set of CSCs.

Referring now to FIG. 10, in a step 1020, a classifier is built using aclassification scheme to distinguish between an object location and anon-object location and using the set of selected features. According toat least one embodiment of the present invention, the step 1020 isperformed using a feature selection method. According to at least oneembodiment of the present invention, the steps 1010 and 1020 are eachperformed using a feature selection method.

In a step 1030, the classifier is applied at a candidate volume todetermine whether the candidate volume contains an object of interest.In at least one embodiment of the present invention, the candidatevolume is obtained by scanning. The candidate volume can be a sub-volumeobtained by scanning. In accordance with exemplary embodiments of thepresent invention, the scanning is performed at a plurality of locationswhile varying scale, rotation and/or aspect ratio.

FIG. 11 is a flowchart showing a method of detection of tumors in 3D CTlung images, in accordance with an exemplary embodiment of the presentinvention. Referring to FIG. 11, in a step 1110, a learning procedure isused for feature selection from a feature set based on an annotatedimage-volume database, generating a set of selected features. Inexemplary embodiments of the present invention, the feature setcomprises 3D features, 2D features, 1 D features, complex features,and/or combinations thereof.

According to at least one exemplary embodiment of the present invention,the step 1110 is performed using a feature selection method. Preferably,the feature selection method is a boosting method. The boosting method,in accordance with exemplary embodiments of the present invention,incorporates feature cost to simultaneously optimize detectionperformance and cost. Detection performance includes, but is not limitedto, classification accuracy. Cost includes, but is not limited to,feature acquisition cost or feature computational cost.

Referring to FIG. 9, in a step 905, the inputs to the boosting methodinclude a set of training samples having a set of features, featurecomputational costs for the set of features, and a pre-determined numberL that denotes the maximal number of strong classifiers.

In a step 910, a set of weak learners is generated based on the set offeatures. In a step 920, a set of FSCs is initialized to null, and a setof CSCs is initialized to contain the set of weak learners.

In a step 930, a selection criterion is used to select the best CSCsfrom the set of CSCs, generating a next set of CSCs. The next set ofCSCs is stored in memory, in a step 940. For each CSC in the next set ofCSCs, if a termination criterion is not satisfied, the CSC is added tothe set of FSCs and removed from the next set of CSCs, in a step 950.Preferably, the termination criterion is based on a validation setand/or a negligible improvement on a class separation margin using atraining set.

In a step 960, it is determined if the magnitude of the set of FSCs isequal to L. In the case when the magnitude of the set of FSCs is equalto L, an output module is performed, in a step 965. In the outputmodule, according to exemplary embodiments of the present invention, thebest strong classifier is selected from the set of FSCs based on abalancing method. Preferably, the balancing method comprises balancingclassification accuracy and feature cost. Feature cost refers to featureacquisition cost and/or feature computational cost.

In the case when the magnitude of the set of FSCs is not equal to L, thenext set of CSCs is re-initialized to null, in a step 970. In a step980, for each the set of CSCs a children function is used to generateone or more new CSC that are added to the next set of CSCs. In anembodiment of the present invention, the children function is used toselect one or more weak learners from the next set of CSCs.

Referring now to FIG. 11, in a step 1120, a classifier is built using aclassification scheme to distinguish between a tumor location and anon-tumor location and using the set of selected features. According toat least one exemplary embodiment of the present invention, the step1120 is performed using a feature selection method. According to atleast one exemplary embodiment of the present invention, the steps 1110and 1120 are each performed using a feature selection method.

In a step 1130, the classifier is applied at a candidate volume todetermine whether the candidate volume contains a tumor. In at least oneexemplary embodiment of the present invention, the candidate volume isobtained by scanning. The candidate volume can be a sub-volume obtainedby scanning. In accordance with exemplary embodiments of the presentinvention, scanning is performed at a plurality of locations whilevarying scale, rotation and/or aspect ratio.

Hereinafter, a computer readable medium including computer code for 3Dobject detection, in accordance with an exemplary embodiment of thepresent invention will be described. The computer readable mediumincludes computer code for using a learning module for feature selectionfrom a feature set based on an annotated image-volume database,generating a set of selected features. The computer readable mediumincludes computer code for building a classifier using a classificationscheme to distinguish between an object location and a non-objectlocation and using the set of selected features. The computer readablemedium includes computer code for applying the classifier at a candidatevolume to determine whether the candidate volume contains an object ofinterest.

Although the processes and apparatus of the present invention have beendescribed in detail with reference to the accompanying drawings for thepurpose of illustration, it is to be understood that the inventiveprocesses and apparatus are not to be construed as limited thereby. Itwill be readily apparent to those of reasonable skill in the art thatvarious modifications to the foregoing exemplary embodiments may be madewithout departing from the spirit and scope of the invention as definedby the appended claims.

1. A method of detecting tumors in 3D medical images comprising: using alearning procedure for feature selection from a feature set based on anannotated medical image-volume database, generating a set of selectedfeatures; building a classifier using a classification scheme todistinguish between tumor location and a non-tumor location and usingthe set of selected features; and applying the classifier at a candidatevolume to determine whether the candidate volume contains a tumor. 2.The method of claim 1, wherein the feature set comprises at least one of3D features, 2D features, 1D features, or complex features.
 3. Themethod of claim 1, wherein at least one of the steps of using a learningprocedure for feature selection or building a classifier is performedusing a feature selection method.
 4. The method of claim 3, wherein thefeature selection method is a boosting method.
 5. The method of claim 4,wherein the boosting method incorporates feature cost to simultaneouslyoptimize detection performance and cost.
 6. The method of claim 5,wherein detection performance comprises classification accuracy, andwherein cost comprises at least one of feature acquisition cost orfeature computational cost.
 7. The method of claim 4, wherein theboosting method comprises the steps of: (a) inputting a set of trainingsamples having a set of features, feature computational costs for theset of features, and a predetermined number L that denotes the maximalnumber of strong classifiers; (b) generating a set of weak learnersbased on the set of features; (c) initializing a set of final strongclassifiers (FSCs) and a set of candidate strong classifiers (CSCs), theset of FSCs being initialized to null, the set of CSCs being initializedto contain the set of weak learners; (d) applying a selection criterion,selecting the best CSCs from the set of CSCs, generating a next set ofCSCs; (e) updating the set of FSCs; (f) if the magnitude of the set ofFSCs is equal to L, performing an output module; and (g) using achildren function, generating at least one new CSC, wherein the at leastone new CSC is added to the next set of CSCs.
 8. The method of claim 7,further comprising: (h) repeating steps (d) through (g) if the magnitudeof the set of FSCs is less than L.
 9. The method of claim 7, wherein theupdating step comprises: determining if a termination criterion issatisfied; and if the termination criterion is not satisfied, for eachCSC in the next set of CSCs, adding a CSC to the set of FSCs andremoving the CSC from the next set of CSCs.
 10. The method of claim 9,wherein the termination criterion is based on at least one of avalidation set or a negligible improvement on a class separation marginusing a training set.
 11. The method of claim 7, wherein the childrenfunction is used to select at least one weak learner from the next setof CSCs.
 12. The method of claim 7, wherein the step of performing theoutput module comprises selecting the best strong classifier from theset of FSCs based a balancing method.
 13. The method of claim 12,wherein the balancing method comprises balancing classification accuracyand feature cost.
 14. The method of claim 1, wherein the candidatevolume is obtained by scanning.
 15. The method of claim 14 whereinscanning is performed at a plurality of locations, varying at least oneof scale, rotation or aspect ratio.
 16. A boosting method for trainingclassifiers that distinguish tumor locations from non-tumor location in3D medical images, wherein said method incorporates feature cost tosimultaneously optimize detection performance and cost, said methodcomprising the steps of: (a) inputting a set of training samples havinga set of features, feature computational costs for the set of features,and a pre-determined number L that denotes the maximal number of strongclassifiers; (b) generating a set of weak learners based on the set offeatures; (c) initializing a set of final strong classifiers (FSCs) anda set of candidate strong classifiers (CSCs), the set of FSCs beinginitialized to null, the set of CSCs being initialized to contain theset of weak learners; (d) applying a selection criterion, selecting thebest CSCs from the set of CSCs, generating a next set of CSCs; (e)updating the set of FSCs; (f) if the magnitude of the set of FSCs isequal to L, performing an output module; and (g) using a childrenfunction, generating at least one new CSC, adding the at least one newCSC to the next set of CSCs.
 17. The method of claim 16, furthercomprising: (h) repeating steps (d) through (g) if the magnitude of theset of FSCs is less than L.
 18. The method of claim 16, wherein updatingthe set of FSCs comprises: determining if a termination criterion issatisfied; and if the termination criterion is not satisfied, for eachCSC in the next set of CSCs, adding a CSC to the set of FSCs andremoving the CSC from the next set of CSCs.
 19. The method of claim 18,wherein the termination criterion is based on at least one of avalidation set or a negligible improvement on a class separation marginusing a training set.
 20. The method of claim 16, wherein the childrenfunction is used to select at least one weak learner from the next setof CSCs.
 21. The method of claim 16 wherein the step of performing theoutput module comprises selecting the best strong classifier from theset of FSCs based a balancing method.
 22. The method of claim 21,wherein the balancing method comprises balancing the classificationaccuracy and feature cost.
 23. A program storage device readable by acomputer, tangibly embodying a program of instructions executable by thecomputer to perform the method steps for detecting tumors in 3D medicalimages, said method comprising the steps of: using a learning module forfeature selection from a feature set based on an annotated medicalimage-volume database, generating a set of selected features; building aclassifier using a classification scheme to distinguish between tumorlocation and a non-tumor location and using the set of selectedfeatures; and applying the classifier at a candidate volume to determinewhether the candidate volume contains a tumor.
 24. The computer readableprogram storage device of claim 23, wherein at least one of using alearning module for feature selection or building a classifier includesperforming a feature selection method.
 25. The computer readable programstorage device of claim 23, wherein the feature selection method is aboosting method.
 26. The computer readable program storage device ofclaim 25, wherein the boosting method incorporates feature cost tosimultaneously optimize detection performance and cost.
 27. The computerreadable program storage device of claim 26, wherein detectionperformance comprises classification accuracy, and wherein costcomprises at least one of feature acquisition cost or featurecomputational cost.
 28. The computer readable program storage device ofclaim 25, wherein the boosting method comprises the steps of: (a)inputting a set of training samples having a set of features, featurecomputational costs for the set of features, and a pre-determined numberL that denotes the maximal number of strong classifiers; (b) generatinga set of weak learners based on the set of features; (c) initializing aset of final strong classifiers (FSCs) and a set of candidate strongclassifiers (CSCs), the set of FSCs being initialized to null, the setof CSCs being initialized to contain the set of weak learners; (d)applying a selection criterion, selecting the best CSCs from the set ofCSCs, generating a next set of CSCs; (e) updating the set of FSCs; (f)if the magnitude of the set of FSCs is equal to L, performing an outputmodule; and (g) using a children function, generating at least one newCSC, wherein the at least one new CSC is added to the next set of CSCs.29. The computer readable program storage device of claim 28, furthercomprising: (h) repeating steps (d) through (g) if the magnitude of theset of FSCs is less than L.
 30. The computer readable program storagedevice of claim 28, wherein the updating step comprises: determining ifa termination criterion is satisfied; and if the termination criterionis not satisfied, for each CSC in the next set of CSCs, adding a CSC tothe set of FSCs and removing the CSC from the next set of CSCs.
 31. Thecomputer readable program storage device of claim 30, wherein thetermination criterion is based on at least one of a validation set or anegligible improvement on a class separation margin using a trainingset.
 32. The computer readable program storage device of claim 28,wherein the children function is used to select at least one weaklearner from the next set of CSCs.
 33. The computer readable programstorage device of claim 28, wherein the step of performing the outputmodule comprises selecting the best strong classifier from the set ofFSCs based a balancing method.
 34. The computer readable program storagedevice of claim 33, wherein the balancing method comprises balancingclassification accuracy and feature cost.
 35. A boosting method fortraining classifiers that distinguish tumor locations from non-tumorlocation in 3D medical images, wherein said method incorporates featurecost to simultaneously optimize detection performance and cost, saidmethod comprising the steps of: (a) inputting a set of training sampleshaving a set of features and feature computational costs for the set offeatures; (b) generating a set of weak learners based on the set offeatures; and (c) building a boosting classifier based on a subset ofthe weak learners to minimize a cost function of the detectionperformance and the feature computational costs.
 36. A program storagedevice readable by a computer, tangibly embodying a program ofinstructions executable by the computer to perform the method steps fortraining classifiers that distinguish tumor locations from non-tumorlocation in 3D medical images, wherein said method incorporates featurecost to simultaneously optimize detection performance and cost, saidmethod comprising the steps of: (a) inputting a set of training sampleshaving a set of features, feature computational costs for the set offeatures, and a pre-determined number L that denotes the maximal numberof strong classifiers; (b) generating a set of weak learners based onthe set of features; (c) initializing a set of final strong classifiers(FSCs) and a set of candidate strong classifiers (CSCs), the set of FSCsbeing initialized to null, the set of CSCs being initialized to containthe set of weak learners; (d) applying a selection criterion, selectingthe best CSCs from the set of CSCs, generating a next set of CSCs; (e)updating the set of FSCs; (f) if the magnitude of the set of FSCs isequal to L, performing an output module; and (g) using a childrenfunction, generating at least one new CSC, adding the at least one newCSC to the next set of CSCs.
 37. The computer readable program storagedevice of claim 36, the method further comprising: (h) repeating steps(d) through (g) if the magnitude of the set of FSCs is less than L. 38.The computer readable program storage device of claim 36, whereinupdating the set of FSCs comprises: determining if a terminationcriterion is satisfied; and if the termination criterion is notsatisfied, for each CSC in the next set of CSCs, adding a CSC to the setof FSCs and removing the CSC from the next set of CSCs.
 39. The computerreadable program storage device of claim 38, wherein the terminationcriterion is based on at least one of a validation set or a negligibleimprovement on a class separation margin using a training set.